Programming Massively Parallel Processors: A Hands-on Approach: The Great Divergence: Performance Trajectories in Computing

The Great Divergence marks a tectonic shift in microprocessor history. Between 2001 and 2009, the performance trajectories of CPUs and GPUs split into the "opening jaws" of a massive capability gap. While traditional CPUs hit the Power Wall—where increasing clock speeds generated unsustainable heat—GPUs leveraged their massive consumer installation base in gaming to fund a pivot toward extreme parallelism.

The Inflection Point

By 2003, the gap began to widen. CPUs remained optimized for sequential logic and latency, while GPUs dedicated their transistor budget to Arithmetic Logic Units (ALUs). This resulted in a transition from Gigaflops (GFLOPS) to Teraflops throughput for GPUs, while CPUs followed a much shallower growth curve.

As of 2009, a high-end Intel i7-960 offered ~70 GFLOPS, while the NVIDIA GTX 280 reached nearly ~933 GFLOPS. This was not just a speed increase; it was a fundamental redesign of how we compute, prioritizing throughput over individual instruction speed.

TERMINAL bash — 80x24

> Ready. Click "Run" to execute.

QUESTION 1

What primary constraint led to the 'Power Wall' for traditional CPUs?

The lack of available memory in the early 2000s.

Thermal and power limitations when increasing clock speeds.

A shortage of transistors on the silicon die.

The transition from 32-bit to 64-bit architectures.

QUESTION 2

According to the Great Divergence, which industry provided the economic engine for GPU R&D?

The Financial High-Frequency Trading market.

The Oil and Gas seismic exploration industry.

The Video Game industry.

The Cryptocurrency mining industry.

QUESTION 3

By 2009, how did the peak performance of an NVIDIA GTX 280 compare to an Intel Core i7-960?

They were roughly equal in throughput.

The CPU was twice as fast as the GPU.

The GPU was nearly an order of magnitude higher (~13x).

The GPU was 100x faster than the CPU.

QUESTION 4

GPUs achieve higher throughput by dedicating more transistors to which component?

Large Level-3 Caches.

Complex Branch Prediction logic.

Arithmetic Logic Units (ALUs).

Instruction Decoders.

QUESTION 5

What is the correct unit for measuring one trillion floating-point operations per second?

GFLOPS.

Teraflops.

Petaflops.

Megaflops.